Skip to content

refactor: consolidate tooling, pipeline blueprint, and asset alignment#1

Open
jdsika wants to merge 20 commits intomainfrom
feature/consolidate-tooling
Open

refactor: consolidate tooling, pipeline blueprint, and asset alignment#1
jdsika wants to merge 20 commits intomainfrom
feature/consolidate-tooling

Conversation

@jdsika
Copy link

@jdsika jdsika commented Mar 9, 2026

Summary

Full consolidation of the hd-map-asset-example repository: tooling, pipeline, generated asset structure, and submodule alignment.

Build System & Tooling

  • Add Makefile as central command center (setup, generate, validate, clean, help)
  • Add pre-commit hooks with SHACL/JSON-LD validation
  • Remove orphan .flake8 (no Python files in root repo)
  • Remove unused pyproject.toml

Asset Structure (EVES-003)

  • Remove hand-crafted asset/ directory (replaced by pipeline-generated output)
  • Add generated/input/ with input_manifest.json (JSON-LD), LFS-tracked simulation data, media, docs, LICENSE
  • Pipeline output goes to generated/output/ (gitignored)

Submodules

  • Consolidate into submodules/ (sl-5-8-asset-tools, EVES)
  • Remove stale ontology-management-base root pointer (nested inside sl-5-8)
  • EVES submodule switched to HTTPS
  • Update sl-5-8-asset-tools pointer to include: pipeline fixes, pathlib migration, examples modernization, external asset references

CI/CD

  • Update release.yml to use make setup && make generate && make validate
  • Track 3D preview JSON files and LFS pointers for binary assets

Documentation

  • Rewrite README.md with setup/usage/structure docs
  • Add copilot-instructions.md

Diff stats

46 files changed, 1194 insertions(+), 11707 deletions(-)

Commits (20)

542c469 chore: update sl-5-8-asset-tools submodule pointer
74f6c09 chore: remove orphan .flake8 (no Python files in root repo)
f6065b3 fix: EVES submodule to HTTPS, validate all generated assets
0692fbd refactor: simplify Makefile - remove duplicated logic, drop make asset zip
4c74dda refactor: cascade make setup to submodule Makefile
349d85a chore: remove dead code and update stale references
a9f143e fix: update filenames to match pipeline output renames
89cb427 chore: update sl-5-8-asset-tools submodule pointer
0931390 docs: fix stale references in copilot-instructions and remove pyproject.toml
09dad83 chore: update sl-5-8-asset-tools submodule pointer
4e3fa1b chore: update submodule pointers
d09e413 chore: update sl-5-8-asset-tools submodule, skip validate when no asset
77b3f39 chore: update sl-5-8-asset-tools submodule pointer
808b370 chore: remove hand-crafted asset/, update submodule pointers
fe0e4d4 docs: clarify internet only needed for geocoding, not SHACL/ontologies
c0a0273 feat: add opendriveconverter as submodule for Python re-implementation
9974692 chore: update sl-5-8-asset-tools submodule (filename + bjson fixes)
5247dc1 feat: track generated/input blueprint with LFS, add 3d_preview assets
0952e3f feat: input_manifest.json pipeline, EVES submodule, cross-platform overhaul
3ad2d8b refactor: consolidate submodules and Makefile as central command center

@jdsika jdsika force-pushed the feature/consolidate-tooling branch 2 times, most recently from 2e95d43 to 54aee8f Compare March 9, 2026 16:36
- Remove root ontology-management-base submodule (now via recursive init through sl-5-8-asset-tools)
- Add Makefile as central command center (OMB pattern)
  - Space-separated subcommands: make asset zip
  - OS detection for cross-platform support (Windows + Unix)
  - Path aliases hiding deep submodule paths
- Update .pre-commit-config.yaml to use make validate hook
- Update release workflow to use make targets exclusively
- Add .github/copilot-instructions.md
- Update README.md with new setup/validation instructions

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
@jdsika jdsika force-pushed the feature/consolidate-tooling branch from 54aee8f to 3ad2d8b Compare March 9, 2026 17:11
jdsika and others added 2 commits March 10, 2026 14:22
…erhaul

Major changes:
- Replace uploadedFiles.json with input_manifest.json (envited-x JSON-LD)
- Add make generate: end-to-end asset creation pipeline with staging,
  validation, QC checks, and zip output to generated/
- Add EVES submodule pinned to asset-preparation-input-manifest branch
- Modernize release.yml: softprops/action-gh-release@v2
- Cross-platform Makefile: Python-based zip/clean, OS detection
- Update sl-5-8-asset-tools to include input_manifest support,
  venv PATH fix for QC executables, and [qc-deps] extras
- Add generated/.gitkeep (contents gitignored, folder tracked)
- Complete README rewrite with pipeline documentation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
- Track generated/input/ as the canonical pipeline blueprint (LFS for
  .png, .pdf, .xodr binaries)
- Add 9 3d_preview JSON files to input manifest for pass-through
- Update .gitignore: only ignore generated/output/, allow input/
- Update Makefile: generate reuses existing blueprint, generate clean
  preserves input/ directory
- Remove obsolete generated/.gitkeep

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
@jdsika jdsika changed the title refactor: consolidate submodules and Makefile feat: consolidate tooling, pipeline blueprint, asset alignment fixes Mar 10, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
jdsika and others added 2 commits March 10, 2026 20:25
Add virtualcitySYSTEMS/opendriveconverter (Java/Gradle) as a submodule
under submodules/opendriveconverter to serve as reference for the
planned Python re-implementation of the OpenDRIVE to GeoJSON converter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Ontology schemas and SHACL shapes are now bundled locally via the
ontology-management-base submodule. Only Nominatim reverse geocoding
still requires an internet connection.

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
jdsika added 14 commits March 10, 2026 22:41
The asset/ folder is replaced by the pipeline-generated output in
generated/output/. Submodule pointers updated to latest feature
branch commits. Removed opendriveconverter submodule.

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
- Update submodule pointer to ruff migration commit
- Make validate gracefully skip when asset/ doesn't exist yet

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
sl-5-8-asset-tools: conditional OMB install for CI
EVES: table alignment fix for eves-007
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Includes wizard_caller rewrite, SPDX license fix, and SD Creation
Wizard submodules.

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
…ct.toml

- copilot-instructions.md: replace deleted asset/ folder references with
  generated/input/ and generated/output/ pipeline workflow
- pyproject.toml: remove (only contained stale black/isort config, no
  Python files in root repo)

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
- manifest_reference.json → manifest.json in Makefile validate + zip targets
- hdmap_instance.json → hdmap.json in Makefile validate + zip targets
- Update copilot-instructions.md to match new naming

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
- Remove unused ASSET_DIR variable from Makefile
- Remove dead staging logic from make generate (require pre-populated input/)
- Update README.md: manifest_reference.json -> manifest.json,
  hdmap_instance.json -> hdmap.json throughout
- Update submodule pointer to sl-5-8-asset-tools

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Instead of duplicating pip install commands for sl-5-8-asset-tools and
ontology-management-base, delegate to the submodule's own make setup/install
targets. Pass VENV and PYTHON as absolute paths so the submodule uses the
root venv. This removes 3 duplicated install blocks and ensures the
submodule's Makefile is the single source of truth for its own deps.

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
…t zip

- Remove make asset zip (pipeline already creates asset.zip)
- Deduplicate generate validate → delegates to make validate
- Simplify validate: use file-redirect instead of subprocess.check_call
- Simplify clean: use native shell commands instead of inline Python
- Update release.yml to use make generate + pipeline-generated zip
- Update copilot-instructions.md for new workflow

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
- Switch EVES submodule URL from SSH to HTTPS for CI compatibility
- Validate target now loops over all asset directories in generated/output/
  instead of only picking dirs[0]

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
Includes pipeline fixes, pathlib migration, examples modernization,
and external asset reference support.

Signed-off-by: jdsika <carlo.van-driesten@bmw.de>
@jdsika jdsika changed the title feat: consolidate tooling, pipeline blueprint, asset alignment fixes refactor: consolidate tooling, pipeline blueprint, and asset alignment Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant